<h1>COCO 2015 Image Captioning Task</h1>
<p align="center"><img src="images/captions-splash.jpg" class="wide80"/></p>

<h1>1. Overview</h1>
<p><b>Update:</b> Challenge winners were announced at CVPR 2015. The COCO caption <a href="https://competitions.codalab.org/competitions/3221" target="_blank">evaluation server</a> remains open. Please submit new results to compare to state-of-the-art methods using several automatic evaluation metrics. The COCO 2015 Captioning Challenge is now, however, complete. Results were presented as part of the CVPR 2015 <a href="http://lsun.cs.princeton.edu/" target="_blank">Large-scale Scene Understanding (LSUN)</a> workshop and are available to view on the <a href="#captions-leaderboard">leaderboard</a>.</p>
<p>This captioning challenge is part of the <a href="http://lsun.cs.princeton.edu/" target="_blank">Large-scale Scene Understanding (LSUN)</a> CVPR 2015 workshop organized by Princeton University. For further details please visit the LSUN website.</p>
<p>The COCO Captioning Challenge is designed to spur the development of algorithms producing image captions that are informative and accurate. Teams will be competing by training their algorithms on the COCO 2014 dataset and having their results scored by <b>human judges</b>.</p>

<h1>2. Dates</h1>
<div class="json">
  <div class="jsonktxt">April 1, 2015</div><div class="jsonvtxt">Training and testing data, and evaluation software released</div>
  <div class="jsonktxt">May  29, 2015</div><div class="jsonvtxt">Submission deadline at 11:59 PST</div>
  <div class="jsonktxt">June  5, 2015</div><div class="jsonvtxt">Challenge results (human judgment) released</div>
  <div class="jsonktxt">June 12, 2015</div><div class="jsonvtxt">Winner presents at the LSUN Workshop at CVPR 2015</div>
</div>

<h1>3. Organizers</h1>
<p>
Yin Cui (Cornell Tech)<br/>
Matteo Ruggero Ronchi (Caltech)<br/>
Tsung-Yi Lin (Cornell Tech)<br/>
Piotr Dollár (Facebook AI Research)<br/>
Larry Zitnick (Microsoft Research)
</p>

<h1>4. Challenge Guidelines</h1>
<p>Participants are recommended but not restricted to train their algorithms on COCO 2014 dataset. The results should contain a single caption for each validation and test image and they must be submitted and publicly published on the <a href="https://competitions.codalab.org/competitions/3221" target="_blank">CodaLab</a> leaderboard. Please specify any and all external data used for training in the "method description" when uploading results to the evaluation server.</p>
<p>By the challenge deadline, both results on the validation and test sets must be submitted to the evaluation server. The results on validation will be public and used for performance diagnosis and visualization. The competitors' algorithms will be evaluated based on the feedback from <b>human judges</b> and the top performing teams will be awarded prizes. Two or three teams will also be invited to present at the LSUN workshop.</p>
<p>Please follow the instructions in the <a href="#download">download</a>, <a href="#format-data">data format</a>, <a href="#format-results">results format</a>, <a href="#guidelines">guidelines</a>, <a href="#upload">upload</a>, and <a href="#captions-eval">evaluation</a> pages. The <a href="https://github.com/tylin/coco-caption" target="_blank">COCO Caption Evaluation Toolkit</a> is also available. The tooklit provides evaluation code for common metrics for caption analysis, including the BLEU, METEOR, ROUGE-L, and CIDEr metrics. Note that for the competition, instead of automated metrics, human judges will evaluate algorithm results.</p>
